Combining and displaying multiple document areas

ABSTRACT

A method includes storing, in response to marking by a user of an area of a displayed document, information on the marked area; displaying an icon representing the marked area; conducting the storing operation and the displaying operation for a different area; and creating, in response to an operation by the user for arranging two or more icons to be in contact with each other, a joined icon by joining the icons together; and combining marked areas represented by the two or more respective icons, according to a state of contact. A corresponding computer program product and computer system are also disclosed herein.

BACKGROUND OF THE INVENTION

The present invention relates generally to a combined-area displaymethod, and more specifically, to a method, a device, and a computerprogram for combining and displaying multiple areas of a document.

In recent years, there have been increased opportunities for browsing adocument on a device having a small display area, such as a smartphone,or a large display area, such as a multi-screen device. Some usersdesire to combine and browse multiple user-desired areas in such anenvironment.

SUMMARY

A method includes storing, in response to marking by a user of an areain the document, information on the marked area; displaying an iconrepresenting the marked area; conducting the storing operation and thedisplaying operation for a different area; and creating, in response toan operation by the user for arranging two or more icons to be incontact with each other, a joined icon by joining the icons together;and combining marked areas represented by the two or more respectiveicons, according to a state of contact. A corresponding computer programproduct and computer system are also disclosed herein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a user interface diagram depicting an example in which adocument is displayed in a display area displayed on a display;

FIG. 2 is an user interface diagram illustrating area selection and anicon of the present invention;

FIG. 3 is a user interface diagram depicting an example of selecting anarea in the display area of the display;

FIG. 4 is a user interface diagram illustrating area selection and anicon of the present invention;

FIG. 5 is a user interface diagram depicting an example of a case ofscrolling a document displayed in the display area;

FIG. 6 is a user interface diagram illustrating an operation forarranging icons to be close to each other;

FIG. 7 is a user interface diagram illustrating generation of a joinedicon;

FIG. 8 is a user interface diagram illustrating an operation forarranging icons to be close to each other;

FIG. 9 is a user interface diagram illustrating generation of a joinedicon;

FIG. 10 is a user interface diagram illustrating generation of a joinedicon;

FIG. 11 a user interface diagram that illustrates examples of generationof a joined icon according to joining direction;

FIG. 12 is a user interface diagram illustrating an operation forchanging a joined icon;

FIG. 13 is a user interface diagram illustrating an operation forrotating the joined icon;

FIG. 14 is a user interface diagram illustrating the area represented byone of the icons in an easy-to-view manner;

FIG. 15 is a user interface diagram illustrating the area represented byeach of the icons in an easy-to-view manner;

FIG. 16 is a block diagram illustrating an example of a functionalconfiguration of an information processor of an embodiment;

FIG. 17 is a block diagram illustrating an example of a hardwareconfiguration of the information processor of the present invention; and

FIG. 18 is a flowchart illustrating an operation example of the presentinvention.

DETAILED DESCRIPTION

A method, computer program product, computer system, and an apparatusfor displaying multiple combined areas in a window, which is a displayarea, according to various embodiments of the present invention arespecifically described below with reference to the drawings. Thefollowing embodiments are not intended to limit the invention describedin the Scope of claims, and it is presumed that not all the combinationsof the characteristic features described in the various embodiments areessential for solutions.

In addition, the present invention can be implemented in a variety ofdifferent modes and therefore should not be construed as being limitedto the described embodiments.

The present invention may be implemented entirely as hardware or assoftware that executes on hardware. For example, as apparent to thoseskilled in the art, the present invention may be partially implementedas a computer-executable program. Hence, the present invention can beimplemented as a hardware embodiment, a combined software and hardwareembodiment, in addition to a computer-implemented method embodiment. Thecomputer program can be stored on any computer-readable recording mediumsuch as a hard disk, a DVD, a CD, an optical storage device, or amagnetic storage device.

Various embodiments of the present invention will be described below indetail with reference to the drawings. These embodiments provide acombined-area display method. A description is given below by taking, asan example, an operation using a pointing device or the like on adisplay of, for example, a personal computer. However, the same appliesto a case of using a finger on a touchscreen of, for example, asmartphone. Note that such an operation by a finger is rather simple toperform when joining or separating icons.

FIG. 17 is a block diagram illustrating a configuration of aninformation processor according to one embodiment of the presentinvention. An information processor 1 according to this embodiment ofthe present invention includes at least a CPU (central processing unit)1701, a memory 1702, a storage device 1703, an I/O interface 1704, avideo interface 1705, a disk drive 1706, a communication interface 1707,and an internal bus 1708 that connects the above-described hardwarecomponents to each other.

The CPU 1701 is connected to hardware components as the above-describedones of the information processor 1 via the internal bus 1708. The CPU1701 controls the operation of each of the hardware components andexecutes various software functions according to a computer program 100stored on the storage device 1703. A load module is loaded into thememory 1702 when the computer program 100 is executed, and temporarydata generated during the execution of the computer program 100, forexample, is stored in the memory 1702.

The storage device 1703 is a built-in, fixed-type storage device thatincludes a ROM. The computer program 100 stored on the storage device1703 is downloaded from a portable recording medium 90, such as a DVD ora CD-ROM, on which information such as programs and data is stored, bythe portable disk drive 1706 and is loaded from the storage device 1703to the memory 1702 at the time of execution. Alternatively, the computerprogram 100 may be downloaded from an external computer connected viathe communication interface 1707.

The communication interface 1707 is connected to the internal bus 1708and is capable of transmitting and receiving data to and from anexternal computer and the like by having a connection to an externalnetwork such as the Internet, a LAN, or a WAN.

The I/O interface 1704 receives data output by an external device. Forexample, when a display 22 is equipped with a touchscreen, signalsgenerated using the touchscreen are received via the I/O interface 1704.The video interface 1705 is connected to the display 22. A window 110 tobe described in an example is displayed on the display 22.

FIG. 1 is a user interface diagram depicting an example of displaying adocument in the window 110 displayed on the display 22 of a computer.The window 110 is generated and displayed by a general application suchas a document editor, an image editor, or a browser. As is widely known,the window 110 is resizable. A description is given below assuming thatthe window 110 is displayed full-screen on the display 22. However, itshould be noted that this embodiment can be implemented withoutdeparting from the essence of the present invention even when the window110 is resized and displayed on a certain part of the display 22.

In FIG. 1, a user specifies an area 150, which the user desires to referto later, in the document displayed in the window 110. In a methodemployed to specify an area, an area is specified by selecting a startpoint 120 and an end point 130 by using a pointer 50 of a pointingdevice, for example. When the area 150 is specified, content informationon the area 150 is stored in the memory 1702 or the storage device 1703in order to be referred to later.

FIG. 2 is an user interface diagram illustrating an icon 200, which is adistinctive feature of the present invention. Upon storing the contentof the area 150, the icon 200, which represents the area 150, isgenerated and displayed. The icon 200 may be displayed at a positionsuch as that in FIG. 2 or a position over the content of the selectedarea.

The icon 200 may be a predetermined icon, or the user may choose theshape of the icon 200. Preferably, the shape of the icon 200 isdetermined according to the content of the area 150. For example, whenthe content of the area 150 includes a text document, an iconrepresenting a book may be used; when the content includes an image, anicon representing a photo may be used; and when the content includes anobject, an icon representing a jewel may be used.

The icon 200 in FIG. 2 has a shape of a puzzle piece, which is used in ajigsaw puzzle and is interlockable. The puzzle-piece shape is used herein order to facilitate the understanding of the joining of icons to bedescribed later. As described above, the icon 200 is generated anddisplayed as an object representing the area 150.

FIG. 3 is a user interface diagram illustrating a state in which, aftergeneration of the icon 200, the user further specifies an area 350,which the user desires to refer to later, in the document displayed inthe window 110 after scrolling the document in the window 110. The area350 is specified by selecting a start point 320 and an end point 330 byusing the pointer 50. When the area 350 has been specified, the contentinformation on the area 350 is stored in the memory 1702 or the storagedevice 1703 in order to be referred to later.

In FIG. 3, look at the icon 200. Although the area 150 is not presentedin the window 110 as a result of scrolling the document, the icon 200 isdisplayed in such a manner as to be connected to the upper side of thewindow 110 via a connecting line 205.

FIG. 4 is an user interface diagram illustrating an icon 400. Uponstoring the content of the area 350, the icon 400, which represents thearea 350, is generated and displayed.

FIG. 5 is a user interface diagram of an example of a screen in the caseof further scrolling the document. In this example, although the area150 and the area 350 are not presented in the window 110, the icon 200and the icon 400, which respectively represent the area 150 and the area350, are displayed in such a manner as to be connected to the upper sideof the window 110 respectively using the connecting line 205 and aconnecting line 405. Such a connecting line is not an essential elementbut has the effect of indicating to the user that the selected area thatis not currently displayed is located before the currently displayedpart of the document.

In the case of scrolling in the opposite direction, each icon isdisplayed in such a manner as to be connected to the lower side of thewindow 110. Preferably, the position at which a connecting line and aside are connected to each other corresponds to the position on thehorizontal axis (x coordinate) at which the corresponding icon isgenerated.

A description is given above of an example of displaying icons in thecase of vertical scrolling. Note that, in the case of horizontalscrolling, each icon is displayed in such a manner as to be connected tothe left side or the right side of the window 110.

A description is given of the combining of icons, which is aparticularly distinctive feature of the present invention, withreference to FIGS. 6 to 11.

First, FIG. 6 is a user interface diagram illustrating an operation forarranging the icon 200 and the icon 400 to be close to each other. Afterthe icon 200 is dragged to a certain position (around the center of thedocument in FIG. 6) by using the pointer 50, the icon 400 is arrangedcloser to the icon 200 in the direction from the right of the icon 200.Preferably, during this operation, each of the connecting line 205 andthe connecting line 405 is displayed in such a manner as to expand orcontract together with the corresponding icon in order to explicitlyindicate that the icon is being dragged.

The shape of the joining icon or the joined icon may be changed so thaticons can be interlocked with each other. Alternatively, an auxiliary(e.g., an arrow or a character) may be displayed near an icon in orderto indicate each possible direction for joining. In addition, anexpected joining position may be displayed. In response to thehorizontal contact between the icon 200 and the icon 400, a joined iconis generated.

FIG. 7 is a user interface diagram illustrating generation of a joinedicon 700 in response to horizontal contact between the icons 200 and400. The icon 700 is a joined icon formed by horizontally joining iconstogether so that the puzzle pieces interlock with each other. At thesame time, the area 150 and the area 350 respectively represented by theicons 200 and 400 are displayed in a horizontally combined manner havinga joining line 710 therebetween. In this state, the joining positions ofthe areas may be determined according to the joining positions of theicons.

200∪400 provided in parentheses for the icon 700 indicates an icongenerated by joining the icon 200 and the icon 400 together.

Preferably, the joined icon is formed in such a way that the shape of anicon is changed before and after joining. As can be seen by carefullyviewing the icon 200 in FIG. 6 and FIG. 7, the shape of the icon 200 isdifferent before and after joining. Changing the shape of the icon asdescribed above can produce a state in which the icons are firmly joinedto each other.

The joined icon may be hidden by a certain operation or after a certainperiod of time. Alternatively, the joined icon may be displayed at areduced size. In the case where the joined icon is hidden, the joinedicon may be displayed again by a certain operation. Alternatively, theicon may be moved to a position at which the icon does not overlap thecontent or may be moved to a desired position. When the operation forseparating the joined icon by using the pointer 50 is performed or whena predetermined termination keystroke combination (e.g.,Ctrl+Alt+BackSpace) is pressed in this state, the joined icon is split,and the display is reverted to the state in FIG. 6.

FIG. 8 is a user interface diagram illustrating an operation forvertically joining the icon 200 and the icon 400. After the icon 200 isdragged to a certain position by using the pointer 50, the icon 400 isarranged closer to the icon 200 in the direction from the bottom side ofthe icon 200. Preferably, during this operation, each of the joiningline 205 and the joining line 405 is displayed in such a manner as toexpand or contract together with the corresponding icon in order toexplicitly indicate that the icon is being dragged. In response to thevertical contact of the icon 200 and the icon 400 with each other, ajoined icon indicating a vertical join is generated.

A predetermined joined icon may be used as a joined icon according tothe types of the icons that are joined together and variations injoining direction, or the shape of a joined icon may be calculated atthe time of contact on the basis of the shapes of the icons that arejoined together and the joining direction in order to generate a joinedicon accordingly.

FIG. 9 is a user interface diagram illustrating that a joined icon 800is generated in response to vertical contact of the icons with eachother. The icon 800 is a joined icon formed by horizontally joining theicons together so that the puzzle pieces interlock with each other. Atthe same time, the area 150 and the area 350 respectively represented bythe icons 200 and 400 are displayed in a vertically joined manner havinga joining line 910 therebetween.

200∪400 provided in parentheses for the icon 800 indicates an icongenerated by joining the icon 200 and the icon 400 with each other. Thejoined icon may be formed in such a way that the shape of an icon ischanged before and after the icon is joined to another icon. As can beseen by carefully viewing the icon 400 in FIG. 8 and FIG. 9, the shapeof the icon 400 is different before and after the icon 400 is joined.

In FIG. 9, it is assumed that an icon 900 representing a differentselected area is connected to the upper side via the connecting line905. FIG. 10 illustrates a state where the icon 900 is horizontallybrought into contact with the joined icon 800.

FIG. 10 is a user interface diagram illustrating a joined icon formed byjoining three icons. The joined icon has a shape obtained by attachingthe icon 900 to the right side of the icon 800 so as to indicate thatthe icon 900 has been horizontally brought into contact with the icon800. At the same time, the area 150 and the area 350 represented by theicons 200 and 400 are displayed in a vertically combined manner havingthe joining line 910 therebetween, and these areas and the arearepresented by the icon 900 are horizontally joined together having ajoining line 1010 therebetween.

FIG. 11 is a user interface diagram that provides examples of a joinedicon generated according to the joining direction of icons. Part A ofFIG. 11 illustrates an example of generation of a joined icon in thecase of vertical joining. Part B of FIG. 11 illustrates an example ofgeneration of a joined icon in the case of horizontal joining.

Part C of FIG. 11 illustrates two examples of generation of a joinedicon in the case where a different icon is horizontally brought intocontact with a joined icon generated by joining two icons. The exampleon the left illustrates a case where an icon is horizontally broughtinto contact with a joined icon at an upper position of the joined icon.The example on the right illustrates a case where an icon is broughtinto contact with a joined icon at a central position of the side of thejoined icon.

In the examples of Part C of FIG. 11, the number of icons after theicons are joined together is an odd number. In such a case, the positionof the icon that has been arranged to be in contact with the other iconsmay indicate the content of the icon to be displayed long.

While an icon can be further added to a joined icon as illustrated inPart C of FIG. 11, an intended icon can be deleted from a joined icon.For deletion, an icon desired to be separated is selected by, forexample, using the pointer 50 with reference to the property of thejoined icon. Alternatively, joining of the icons is terminated by anoperation for arranging, to a certain distance, the icon that has beenarranged to be in contact with the other icon. Thereafter, the areasrepresented by the remaining icons are rejoined so as to correspond tothe joined icon after deletion.

FIG. 12 is user interface diagram illustrating an operation for changinga joined icon. In FIG. 12, each of the icons that have been joinedtogether has a fan shape instead of the shape of a puzzle piece in FIG.11. Thus, each of the icons that have been joined together can have anyof various shapes without departing from the essence of the presentinvention.

In FIG. 12, a joined icon 1200 generated by joining three icons isillustrated. In addition, an area 1050, which is combined so as tocorrespond to the joining state of the three icons is illustrated. Thisstate is changed to that illustrated in FIG. 13 when an operation forrotating the joined icon 1200 90 degrees in the clockwise direction byusing the pointer 50 is performed in this state.

In FIG. 13, the area 1050 is displayed in a rotated manner as a resultof the rotation of the joined icon 1200. The content of each area isdisplayed so as to fit in a corresponding displayed area. Although FIG.13 illustrates an example of 90-degree clockwise rotation,counterclockwise rotation is also possible. In addition, rotation to anyangle is possible when it is not necessary to consider the ease ofcontent viewability. In the case of rotation of 45 degrees, although notillustrated, the combined areas are formed by areas inclined at 45degrees with respect to the rectangular areas.

FIG. 14 user interface diagram illustrating a method of displaying thearea represented by an icon in an easy-to-view manner. FIG. 14illustrates an example in which fan-shaped icons 200, 400, and 900 arerespectively connected, via the connecting lines 205, 405, and 905, tothe upper side of the window 110. When the pointer 50 is brought closeto the icon 200, the content of the area 150 represented by the icon 200is displayed near the icon 200.

The content to be displayed near each icon is text selected from thetext document in the area at the time of generating the icon. In thisway, the areas represented by each icon can be understood instantly.

FIG. 15 is a user interface diagram illustrating another method ofdisplaying the area represented by an icon in an easy-to-view manner. InFIG. 15, icons 200, 400, and 900 are respectively connected, via theconnecting lines 205, 405, and 905, to the upper side of the window 110as in FIG. 14, and the content of the area represented by each icon isgenerated and displayed at a reduced size as the icon. This method ismore effective when the content is an image than when the content istext.

As another example, when the pointer 50 is hovered above an icon, thetext stored at the time of generating the icon may be displayed.Alternatively, a thumbnail of the content in the area may be displayed.Alternatively, a user may be able to input text, and the text may bedisplayed.

FIG. 16 is a block diagram illustrating an example of a functionalconfiguration of an information processor 1. As illustrated in FIG. 16,the information processor 1 includes an operation detection unit 1610, amarked-area management unit 1620, an icon generation unit 1630, anicon-joining management unit 1640, a marked-area combining unit 1650, amarked-area information storing unit 1660, and an icon storing unit1670.

The operation detection unit 1610 detects whether a user has marked anarea. Upon detection of a marked area, the marked-area management unit1620 stores the information on the marked area in the marked-areainformation storing unit 1660 and instructs the icon generation unit1630 to generate an icon. The icon generation unit 1630 generates anicon on the basis of the stored marked-area information, stores thegenerated icon in the icon storing unit 1670, and instructs the display22 to display the icon.

In addition, the operation detection unit 1610 detects whether anoperation for arranging icons to be in contact with each other has beenperformed. When icons have been brought into contact with each other,the icon-joining management unit 1640 instructs the icon generation unit1630 to generate a joined icon. The icon generation unit 1630 generatesa joined icon based on the contact state of the icons, stores the joinedicon in the icon storing unit 1670, and instructs the display 22 todisplay the joined icon.

As a usage example of a stored joined icon, the joined icon is retrievedby using a specified keystroke combination in order to restore the pastdisplay of combined areas.

Upon generation of the joined icon, the icon-joining management unit1640 instructs the marked-area combining unit 1650 to combine thecorresponding areas. The marked-area combining unit 1650 combines theareas according to the joining state of the joined icon and instructsthe display 22 to display the combined areas.

FIG. 18 is a flowchart illustrating an operation example of the presentinvention. The operation process is assumed to start at the time whenthe operation detection unit 1610 has received an instruction to combineareas. This operation process may be started in such a way that anapplication intercepts an operation performed to specify an area or maybe performed in response to an instruction from an agent for monitoringkeystroke operation, in a state where the agent has been activated.

When the operation process is started, detection of a marked area isperformed in step 1802. When a marked area is detected, information onthe marked area is stored in the marked-area information storing unit1660 in step 1804. Then, in step 1806, an icon representing the markedarea is generated.

The icon may be appropriately chosen from multiple predetermined iconsor may be automatically generated on the basis of the stored informationon the marked area. Then, in step 1808, the generated icon is displayed.

Next, in step 1810, it is determined whether icons have been broughtinto contact with each other. When no contact is detected, the processreturns to step 1802. When contact of icons is detected in step 1810,the icons are displayed in a joined manner in step 1812.

The icons are joined with reference to the contact state, and a joinedicon is generated on the basis of the direction in which the icon hasbeen brought into contact with the other. The joined icon is generatedon the basis of the shape of the joining-side icon, the information onthe area represented by the icon, and the contact state and isdisplayed.

When the joined icon is generated, the marked areas are combined anddisplayed in such a manner as to correspond to the joined icon (i.e., tocorrespond to the contact state) in step 1814.

In step 1816, it is determined whether a change operation has beenperformed on the joined icon. The change operation is, for example,rotation of the joined icon or deletion of any of the individual iconsthat have been joined together. When a change operation is detected,changes are made to the joined icon and the combined areas according tothe change in step 1818, and the process returns to step 1812.

When no change operation is detected in step 1816, it is determined instep 1820 whether an instruction to terminate the joining has beenissued. When no such instruction is detected, the process returns tostep 1802.

When an instruction to terminate the joining is detected in step 1820,joining of the icons and displaying of the areas in a combined mannerare terminated in step 1822, and the process is terminated. Theabove-described area-combining process may be terminated at any time bya predetermined keystroke operation.

A number of application examples of the present invention areconceivable without departing from the essence of the present invention.For example, multiple users can browse an information material, anarticle, a development document, or the like by exchanging stored iconsand the information on the corresponding areas between differentterminals via the communication interface 1707. In addition, documentsof different content types such as an image, an object, and XML, inaddition to text, can be combined and displayed without anyrestrictions, consequently increasing the efficiency of editing work anddevelopment work.

The present invention is also applicable to SNSs. In a case where SNSusers sharing and following marked areas and joined icons follow an SNSon a sentence-by-sentence basis, it is possible to provide sentences andelements that are attracting particular attention, to the users by usingcombined areas displayed in an easy-to-view manner Note that the presentinvention is not limited to the above-described embodiments. Variouschanges and improvements are possible within the gist of the presentinvention.

What is claimed is:
 1. A device for combining and displaying a pluralityof areas of a document, the device comprising computer hardwarecomponents configured to perform a method comprising: storing, inresponse to a user marking an area of the document, information on themarked area; displaying an icon representing the marked area; conductingthe storing operation and the displaying operation for a different area;and forming, in response to an operation by the user for arranging twoor more icons to be in contact with each other, a joined icon by joiningthe icons together; and combining marked areas represented by the two ormore respective icons, according to a state of contact.
 2. The device ofclaim 1, wherein the method further comprises changing the joined iconand changing a combining state of the marked areas in response to anoperation by the user for separating part of the joined icon, enablingcontact of a new icon, or rotating the joined icon.